The present invention relates to a device for image processing and specifically to a method and device for transferring images.
Techniques for enabling compression of moving picture with high compression rates have been developed and employed in digital cameras and digital video cameras. A generally known moving picture compression scheme is MPEG (Moving Picture Experts Group) standardized by ISO (International Organization for Standardization).
MPEG deals with bit streams obtained as a result of coding which utilizes intraframe correlation on a macroblock by macroblock basis where each macroblock consists of NX (horizontal) by NY (vertical) pixels in an image (Intra-coding) or coding which utilizes interframe correlation (Inter-coding). An image structured only by Intra-coding is called an “I-picture”, and an image structured by Intra-coding and Inter-coding in a mixed manner is called a “P-picture” or “B-picture”.
MPEG bit streams produced at the time of encoding and MPEG bit streams input at the time of decoding are subjected to various processes, including inverse quantization, inverse DCT, motion compensation, etc., and a resultant reconstructed image is stored in a temporary data storage. The stored image is read out as a reference image.
Now, consider a case where an SDRAM (synchronous dynamic random-access memory) is used as the temporary data storage. In encoding and decoding processes, transfer of reconstructed images and reference images occurs many times in this SDRAM. A generally employed method for the transfer with a high rate is transferring a plurality of pixels at one time through an expanded data bus width of the SDRAM. For example, if the data bus width of the SDRAM is 4 times that of pixel data, data consisting of 16 (horizontal) by 16 (vertical) pixels as shown in FIG. 11 is transferred to the SDRAM on a data pack by data pack basis where each data pack (a unit of data for transfer or “data transfer unit”) consists of 4 horizontally consecutive pixels. Namely, data consisting of 4 (horizontal) by 16 (vertical) data packs as shown in FIG. 13 is transferred to the SDRAM on a data pack by data pack basis. In this case, the transfer cycles are reduced to a ¼ of those required for pixel-by-pixel transfer.
Specifically, now consider transfer of the luminance signal of a reconstructed image having the size of one macroblock. FIG. 51 illustrates an example of mapping of data stored in the SDRAM. For example, data packs P0, P1, P2, P3, . . . , P60, . . . , P63 shown in FIG. 13 are transferred to addresses of the SDRAM, SD (ADXR, ADYR), SD (ADXR+1, ADYR), SD (ADXR+2, ADYR), SD (ADXR+3, ADYR), . . . , SD (ADXR, ADYR+15), . . . , SD (ADXR+3, ADYR+15) shown in FIG. 51 as follows.
First, after the row address is set to ADYR, data packs P0 to P3 are transferred. Then, after the row address is set to ADYR+1, data packs P4 to P7 are transferred. Then, after the row address is set to ADYR+2, data packs P8 to P11 are transferred. Then, data packs P12 to P63 are transferred while the row address is changed in the same way.
An example of a decoding device wherein a line memory is used to reduce transfer of reference images is disclosed in Japanese Laid-Open Patent Publication No. 2002-152756.
Transfer of data packs P0 to P7 of FIG. 13 is more specifically described. FIG. 52 is a timing chart for the SDRAM in writing of data packs as in FIG. 51. It is assumed herein that the burst length (BL) of the SDRAM is 4.
First, row address ADYR is set at time t0 (“act” command), column address ADXR is set at time t1 (“write” command), and data packs P0 to P3 are continuously written.
Then, the SDRAM executes “pre” command at time t5 for switching the row address and then sets the next row address ADYR+1 at time t6 (“act” command). At time t7, column address ADXR+1 is set (“write” command), and data packs P4 to P7 are continuously written. Hereinafter, the same procedure is repeated.
Herein, the time required for transferring a line of the reconstructed image between the current “act” command and the next “act” command is 8 cycles. Only 4 out of 8 cycles are used for the data transfer process, while the other cycles are consumed for the address setting process. Namely, every data transfer of 4 data packs entails address setting which consumes 4 cycles. This is quite inefficient.
Next, reading of a reference image is described. FIG. 53 is a timing chart for the SDRAM in reading of data packs stored as shown in FIG. 51. In an instance described herein, the reference image to be read has one macroblock size, i.e., consists of 4 (horizontal) by 16 (vertical) data packs. This read process consumes 8 cycles for transfer of 4 data packs, and 4 out of 8 cycles are used for the address setting process.
FIG. 54 schematically illustrates transfer of 17 pixels. The reference image sometimes consists of 17 (horizontal) by 17 (vertical) pixels. Since every 4 horizontally consecutive pixels constitute a single data pack, transfer of one row, i.e., transfer of 17 pixels, requires 5 data packs. For example, in the case where the 4n-th pixel subsequent to the leftmost pixel included in data P0 of reconstructed image R0 (n is an integer) is selected as the start position of the read operation, reading of 5 data packs is enough for obtaining all the information of 17 pixels (see FIG. 54A) Likewise, in the case where the (4n+1)th, (4n+2)th or (4n+3)th pixel subsequent to the leftmost pixel is selected as the start position of the read operation, reading of 5 data packs is enough for obtaining all the information of 17 pixels (see FIGS. 54B, 54C and 54D, respectively).
FIG. 55 is a timing chart for the SDRAM for reading of data packs stored as shown in FIG. 51 where the reference image consists of 17 (horizontal) by 17 (vertical) pixels. Since vertically-aligned pixels do not constitute a data pack, reading of data of 17 lines is enough for obtaining information of 17 vertically-aligned pixels. Thus, 5 (horizontal) by 17 (vertical) data packs are read out.
Herein, the burst length (BL) of the SDRAM is 4, and therefore, a single “read” command only enables reading of 4 data packs. Thus, a “read” command is executed again at time t4 for enabling reading of 5 consecutive data packs from a region of the same row address, and at time t5, a “pre” command is executed for changing the row address.
As seen from the above, the transfer of the reference image also requires address setting as many times as the number of lines, which is quite inefficient as is the transfer of a reconstructed image. The number of read/write transfer cycles is calculated by the following operation:(Number of accesses to same row address of SDRAM+Number of cycles for address setting of SDRAM)×Number of changes of row address of SDRAM  (C1)Of these cycles, the number of cycles required for address setting of the SDRAM is:Number of cycles for address setting of SDRAM×Number of changes of row address of SDRAM  (C2)
As to the SDRAM used herein, it is understood from FIG. 52 (writing) and FIG. 53 (reading) that “number of cycles for address setting of SDRAM”=4 cycles. Since for each macroblock “number of accesses to same row address of SDRAM”=4 and “number of changes of row address of SDRAM”=16, the number of cycles for writing a reconstructed image of one macroblock is (4+4)×16=128. It is thus understood that 2 (=128/64) cycles are necessary for transfer of one data pack.
Now, consider transfer of a color difference signal of 4:2:0 format in the same way. The luminance signal for one macroblock consists of 16 (horizontal) by 16 (vertical) pixels, and accordingly, the blue color difference signal and red color difference signal each consists of 8 (horizontal) by 8 (vertical) pixels for one macroblock. In the case where every 4 horizontally consecutive pixels constitute a data pack, the number of data packs for one of the color difference signals is 2 (horizontal) by 8 (vertical) data packs. With “number of accesses to same row address of SDRAM”=2 and “number of changes of row address of SDRAM”=8, the number of transfer cycles is calculated using formula (C1), resulting in (2+4)×8=48 cycles. Transfer of both of the color difference signals requires 96 cycles (=48×2). Thus, it is understood that transfer of 1 data pack requires 3(=96/32) cycles.
These periods increase relative to the frame size. For example, an image of HD (high definition) size has 2,073,600 pixels for one frame and, therefore, when 60 frames are processed in every second, the total number of pixels to be processed is 124,416,000 (=2,073,600×60) pixels per second. To achieve high-speed transfer, with the above-described data packing (4 pixels constituting one data pack), the number of access data packs to the SDRAM is 31,104,000 (=124,416,000/4) data packs.
Now, consider transfer of such data packs to the SDRAM. Assuming a system where the number of transfer macroblocks of each of a reconstructed image and a reference image is one for one macroblock, data packs to be transferred is as much as a total of 62,208,000 (=31,104,000×2) data packs per second. Since the SDRAM access cycle of the luminance signal is “one data pack=2 cycles”, transfer of the whole luminance signal of one second requires 124,416,000 (=62,208,000×2) cycles. A half of these cycles are consumed by address setting of the SDRAM.
Calculation of transfer of the color difference signal is now described. The amount of information of color difference signal of 4:2:0 format (blue color difference signal+red color difference signal) is a half of the amount of information of the luminance signal, and therefore, the number of data packs which are to be transferred to the SDRAM is 31,104,000 (=62,208,000/2) per second. Since the SDRAM access cycle of the color difference signal is “one data pack=3 cycles”, transfer of the whole color difference signal of one second requires 93,312,000 (=31,104,000×3) cycles. A ⅔ of these cycles are consumed by address setting of the SDRAM.
Thus, the total of luminance signal and two color difference signals requires 217,728,000 cycles per second. Of these cycles, 124,416,000 cycles are used for address setting of the SDRAM. This is quite inefficient transfer. As a matter of course, high speed transfer requires a high speed operation of the SDRAM and hence entails large power consumption.
FIG. 56 shows an example of mapping of data stored in the SDRAM where fields of the data are allocated to different regions of the SDRAM. In the interlace process, a macroblock is divided into two fields, the top field consisting of pixels of odd-numbered lines (P0, P1, P2, P3, P8, P9, P10, P11, . . . , P59) and the bottom field consisting of pixels of even-numbered lines (P4, P5, P6, P7, P12, P13, P14, P15, . . . , P63) as illustrated in FIG. 26.
In many cases, the both fields are stored in the SDRAM such that the top field and the bottom field are stored in different regions of the SDRAM as illustrated in FIG. 56. This is because, in an operation of reading from the SDRAM (transfer of the reference image), a transfer request is issued for every field, and in such a case, mixture. of data of both fields inhibits highly efficient transfer.
As for the example of FIG. 56, the number of cycles required for transfer is calculated using formula (C1). Since for each macroblock “number of accesses to same row address of SDRAM”=4 and “number of changes of row address of SDRAM”=8, (4+4)×8=64 cycles are necessary for each macroblock, and hence, 128(=64×2) cycles are necessary for the total of both fields. Thus, 2(=128/64) cycles are necessary for transfer of one data pack. This transfer is inefficient as is the progressive process illustrated in FIG. 51.